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ABSTRACT 

Several approaches to machine analysis of remotely sensed data have 
been developed over the past decade, and the remote sensing data analyst 
is faced with selecting which analysis approach might perform best for 
a given problem. The overall objective of this study was to apply and 
evaluate several currently available classification schemes tor crop 
identification. The approaches examined were: (1) per point Gaussian 

maximum likelihood classifier, (2) per point sum of normal densities 
classifier, (3) per point linear classifier, (4) per point Gaussian 
maximum likelihood decision tree classifier, and (5) texture sensitive 
per field Gaussian maximum likelihood classifier. 

Seven agricultural data sets were used in the study and were selected 
to sample variability in soils, climate, and agricultural practices. 

Five data sets were from the U.S. Corn Belt, and two were from the U.S. 
Great Plains. 

Test site location and classifier both had significant effects on 
classification accuracy of small grains; classifiers did not differ 
significantly in overall accuracy. The majority of the difference among 
classifiers was attributed to training method rather than to the classi- 
fication algorithm applied. The complexity of use and computer costs for 
the classifiers also varied significantly. 


Dr. Akiyama was a Visiting Scientist at the Laboratory for Applications 
of Remote Sensing at the time this work was conducted. 
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I. 


INTRODUCTION 


Over the past several years, the potential utility of remotely sensed 
data to survey and monitor agricultural crops and soils has been Increasingly 
recognized. The use of a per point maximum likelihood classifier In tlie 
Corn Blight Watch Experiment during 1971 was the first attempt at large 
scale application of digital classification of remotely sensed multispectral 
data [1]. This was followed in 1973-75 by the Crop Identification 
Technology Assessment for Remote Sensing (C'^i’-iS) project for corn and 
soybeans In Indiana and Illinois using Lam ■ - l MSS data [2], Since then 
extensive research has been devoted to wheat Inventory with the Large Area 
Crop Inventory Experiment (LACIE) during 1973-78 [3]. Currently, Interest 
has been directed towaru analysis of multicrop areas, with corn and soybeans 
being the major crops of interest. 

To support these efforts utilizing satellite remotely sensed data, 
several numerical analysis schemes have been developed and Implemented 
at numerous university, business, and government facilities in the United 
States and abroad. Some methods developed were successful for identification 
of agricultural crops, while otliers were not as successful. The remote 
sensing data analyst, therefore, must determine wlilch analysis approach or 
algorithm might perform best for a given problem. Numerous studies have 
evaluated the performance ol a given classifier, but relatively few studies 
have objectively compared the performance of several approaches for a 
given problem. 

The overall objective of this study was to apply several currently 
available classification schemes and to evaluate their performance on 
several agricultural data sets. The data sets were selected to include 
corn and soybeans; winter wheat; and spring wheat as the major crops. 
Classification accuracy for test fields, ease of analyst use, and computer 
time required were compared for .he different classifiers and data sets. 

II. EXPERIMENTAL APPROACH 

Test sites were selected fnm three major data sets; CITARS data 
from 1973 over Illinois and Indiana [2]; LACIE data from 1976 over the 
U.S. Great Plains [l]; and multicrop data from 1978 [4]. An 8 x 24 
kilometer (5 x 15 mile) area in Fayette County In south central Illinois 
was used from the CITARS data set. A 9.3 x 11.1 kilometer (5x6 
nautical mile) area was selected in each of Foster County, North Dakota 
and Grant County, Kansas, from the LACIE data. Four segments, each 9.3 
X 11. I km, were selected from the multicrop data: Pottawattamie and 

Shelby Counties in west central Iowa, Tippecanoe County In west central 
Indiana, and Iroquois County in east central Illinois. 

The segments sample several major crops: winter wheat In Kansas: 

spring wheat in North Dakota; and corn and soybeans In Indiana, Illinois 
and Iowa. The Corn Belt segments were located in two distinct regions 
to sample variability in soils, climate, and agricultural practices. 

Both areas are Intensively cropped, with corn and soybeans being the 
predominant agricultural crops. Ground reference data and field maps as 
well as cloud-free mul ti temporal ly registered digital Landsat MSS data 
were available over those sites. 


Four acquisition dates were selected for analysis from the most cloud- 
free, least noisy, and best registered acquisitions which temporally 
sampled the crop calendar to maximize crop development differences (Table 1). 
For the Com Belt segments, an attempt was made to obtain a spring 
acquisition to better separate winter small grains, trees, and peraanent 
pasture from row crops. An acquisition after corn had tasseled was 
Included to separate corn and soybeans. 

Since classification costs would be too high if all lb bands of 
data were used, classifications were performed using four bands selected 
to maximize the average transformed divergence between pairs of classes [s]. 
The acquisition dates and spectral bands selected are shown in Table 2. 

Five classifiers, implemented on an IBM 370/148 computer at the 
Laboratory for Applications of Remote Sensing (LARS), Purdue University, 
were selected for study: 

1. CLASSIFYPOINTS is a per point Gaussian maximum likelihood 
classifier. It is a processor in LARSYS, a remote sensing 
data analysis system developed at LARS £b]. 

2. CLASSIFY' implements a sum of normal densities maximum likelihood 
classification rule which first assigns each pixel into an 
information category and then assigns the pixel to a spectral 
subclass within that category. It is a processor in EODLARSYS, 
developed at NASA/Johnson Space Center [?]. 

). MINIMUM DISTANCE Is a linear classification rule which assigns 
each pixel to tlie class whose mean Is closest in Euclidean 
distance [S]. It Is a processor in LAKSYS. 

4. The LAYERED classifier is a multistage decision procedure! [9]. 

It utilizes decision tree logic with an optimum subset of 
features at each tree node to classify each pixel using a 
Gaussian maximum likelihood decision rule. LAYERED is also 

a processor in LARSYS. 

5. ECHO (Extraction and Classification of Homogeneous Objects) 
utilizes both spectral and local spatial information [lO], 

Statistical tests are used to segment the image into homogeneous 
regions and each region is then classified using a Gaussian 
maximum likelihood sample classification rule. It was also 
developed at LARS and is part of LARSYS. 

In order to insure that differences in classification accuracies were 
the result of classifier differences and not training methods, the same 
set of training statistics was used for all classifiers. Training fields 
were selected to represent the classes of Interest: corn, soybeans, and 

others in tlie Corn Belt segments and small grains and others in tlie Great 
Plains segments. Tliese fields were clustered to develop weans and 
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Table la. Multitemporal Data Set Composition for the Corn and Soybean lest Sites 





Table 2. Spectral Banda Uaed In CloHHiricatlon. 


Test Site 

Lands at 

Acquisition Date 

Spectral Bands Selected 



pm 

Fayette 

6/10 

.6-. 7 


6/29 

None 


7/17 

.6-. 7, .8-1.1 


8/21 

.6-. 7 

Pottawattamie 

6/16 

.8-1.1 


7/23 

.6-. 7, .8-1.1 


9/6 

.7-. 8 


9/24 

None 

Shelby 

6/16 

.6-. 7 


7/23 

.8-1.1 


8/9 

.8-1.1 


9/24 

.8-1.1 

Tippecanoe 

6/10 

00 

1 

■ 


7/26 

.8-1.1 


8/21 

.7-. 8 


9/26 

tJonc 

Iroquois 

6/12 

.7-.« 


8/15 

.8-1.1 


8/31 

.8-1.1 


9/28 

.6-. 7 

Grant 

3/13 

.8-1.1 


5/15 

.6-. 7 


6/2 

.6-. 7 


7/8 

.6-. 7 

Foster 

5/26 

.7-. 8 


6/30 

.7-. 8 


7/19 

.6-. 7 


8/24 

.8-1.1 




covariances to define spectral subclasses for each of the classes of Interest. 
Since CLASSIFY was designed as part of an automated analysis procedure 
without analyst intervention, a training method (referred to as ISOCLS) 
using a random selection of Individual pixels to define initial cluster 
seeds for clustering the entire area Is generally used In conjunction with 
that algorithm. Both training methods were used In CLASSIFY. 

The Fayette County site had reference data over approximately twenty- 
five percent of its area, while reference data were available f"r the 
entire area for the other sites. These data were sampled to define training 
and test data. Half of the selected fields were used for training the 
classifiers, and the remaining half were set aside for testing the classifi- 
cation results. Training was based on 1.6% of the area in the Fayette 
site, and between 3.5 and 7.5% in the other sites. 

III. RESULTS AND DISCUSSION 

The results of the classifications (Table 3) were analyzed to assess 
the effects of segment and classifier on classification accuracy. Segment- 
to-segment variability was highly significant (p<0.01). Segment variability 
was attributed to factors other than the classifier selected, including 
spectral data quality and scene characteristics. 

Several factors contributed to the lower classification accuracies 
obtained in Fayette County; (1) the quality of multitemporal registra- 
tion was only marginal, (2) the acquisitions for Fayette were not as 
well distributed throughout the growing season as in the other counties, 

(3) less training data were available for the Fayette site, and (4) the 
training data were not as well distributed or representative. 

Pottawattamie and Tippecanoe Counties had larger Held sizes, 
accounting in part for the relatively accurate classification. Shelby 
County contained more confusion crops, including sorghum and spring oats, 
and had smaller field sizes than the other counties. Iroquois County had 
very few confusion crops and was almost entirely corn and soybeans. This 
crop distribution made it difficult to obtain training for cover types other 
than corn and soybeans. 

There was no significant difference amrng classifiers in percent 
correct classification of corn, soybeans, or other in the five Corn Belt 
segments. In addition, there was no significant difference in overall 
accuracy using all seven segments. The sum-of-normal-denslties classifier 
using LARSYS statistics, however, gave significantly higher small grain 
classification accuracy (about 2% classification improvement). 

Table 4 shows the percent correctly classified averaged over all 
segments for the different cover types. Tlie performance of the ECHO 
classifier was not as high as anticipated. This is probably due to the 
fact that the ECHO classifier requires the analyst to set parameters defining 
cell size and homogeneity factors, and the optimal settings have not been 
defined. Although differences were nonsignificant overall, the LARSYS 
training method provided a consistent improvement over the ISOCLS training 
method in six of the seven segments. In conclusion, given a set of training 
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Table 3. Comparison of Classifier Performance (Percent Correct Classification) 
by Test Site. 


CLASSIFIER 


TEST 

SITE CLASS 

MINIMUM 

DISTANCE 

CLASS irr 

POINTS 

LAYERED 

ECHO 

cussirr 

Using 

ISOCLS 

Stata 

CLASSin 

Using 

LAXSYS 

State' 

TEST 

SITE 

Average 

Fayetta. IL 

Con 

Bl.f 

81.2 

63.9 

77.3 

77.3 

78.9 

76.8 

Soybeans 

82.0 

77.0 

76.8 

70.7 

49.7 

79.0 

72.5 

Other 

85. S 

88.6 

91.3 

87.8 

58.8 

85.6 

82.9 

Overall 

83.5 

83.0 

80.5 

79.5 

61.1 

81.6 

78.2 

Fottawattaelat lA 

Con 

■ 98.7 

97.2 

95.7 

98.2 

93.0 

98.4 

96.9 

Soybeans 

92.0 

89.8 

92.3 

90.2 

86.5 

89.3 

90.0 

Othar 

85.3 

98.0 

97.5 

97.1 

92.1 

98.4 

94.7 

Overall 

94.9 

94.7 

94.7 

95.4 

90.6 

95.3 

94.3 

Shelby, lA 

Con 

97.1 

95.1 

94.5 

96.1 

82.8 

95.9 

93.6 

Soybeana 

89.3 

92.9 

98.2 

95.4 

98.0 

98.0 

95.3 

Other 

75.5 

83.7 

88.2 

79.4 

78.7 

79.7 

80.9 

Overall 

90.0 

91.7 

93.3 

91.5 

83.9 

92.1 

90.4 

Tippecanoe, HI 

Con 

93.7 

89.9 

91.5 

86.4 

99.4 

93.1 

92.3 

Soybeans 

97.6 

98.2 

94.9 

98.0 

95.1 

98.4 

97.0 

Other 

94.3 

96.7 

100.0 

96.7 

69.9 

96.7 

92.4 

Oversll 

95.5 

94.3 

94.0 

92.7 

94.2 

95.9 

94.4 

Iroquois, IL 

Con 

88.1 

79.5 

91.0 

79.3 

89.9 

92.8 

85.1 

Soybeans 

82.8 

BS.2 

78.1 

83.6 

78.8 

86.3 

82.5 

Other 

76.4 

72.7 

0.0 

72.7 

74.5 

75.0 

61.9 

Oversll 

84.9 

82.1 

80.5 

81.2 

83.6 

84.2 

82.8 

Fostsr, MD 

Snail Creina 

96.1 

95,4 

94.6 

94.8 

93.6 

97.3 

95.3 

Other 

73.3 

77.1 

77.0 

77.6 

70.5 

82.3 

76.3 

Overall 

82.7 

84.7 

84.3 

84.8 

81.3 

89.3 

84.5 

Grant, ES 

Snail Grains 

96.9 

96.7 

97.6 

96.5 

94.6 

98.7 

96.8 

Other 

91.8 

83.2 

89.3 

79.2 

92.0 

80.2 

86.0 

Oversll 

93.1 

86.5 

91.4 

83.5 

92.6 

84.8 

88.6 

^Training nethod generally used with CLASSIFY. Uaes a 

randon selection of 

Individual pixels to 

define Initial 

cluster seeds for clustering 

the entire 

area* 







^Tralnlns aethod uaed with all other claaatficra. Training flelda were cluatered to develop aeana and covarlancea 
to define apectral aubelaaaca for each of the claaaea of Intercat. 



8taLlNticH capable of producing high level classification results, the 
choice of classification algorithm for differentiation of corn and soybeans 
from other cover types makes relatively little difference. 

TVo additional comparisons of the classification schemes were considered 
the ease of use of the classification method and the computer time needed 
for each classifier. The classification schemes varied considerably In 
ease of use. In Increasing order of complexity the classifiers were found 
to be: (1) MINIMUM DISTANCE, (2) CLASSIFYPOINTS, (3) CLASSIFY, (4) ECHO, 

and (5) LAYERED. The MINIMUM DISTANCE and CLASSIFYPOINTS classifiers were 
almost Identical In ease of use. 

CLASSIFY was designed as part of a total analysis scheme In which 
participation of the analyst Is minimized In the clustering and definition 
of training statistics with control provided by a predefined set of 
analysis parameters. Although the classifier itself is not extremely 
complex, the training procedure typically used In this scheme involves a 
large number of parameters about which little Is known. 

ECHO utilizes both temporal and spatial Information. The complexity 
of use for ECHO arises from the necessity of setting the parameters for 
cell liomogeneity testing and cell size. The expertise of the analyst Is 
essential In setting the parameters with regard to data set used. The 
ECHO classifier Is, however, one of the few available classifiers that 
utilize spatial as well as spectral information In the classification 
process . 

LAYERED implements a per point Gaussian maximum likelihood decision 
tree logic which requires the additional step of designing the decision 
tree. The decision tree is designed by obtaining class moans and 
covariance matrices for all classes and using a feature selection algorithm 
to determine an optimal subset of features to be used at each node of the 
decision tree. The time needed by the analyst to design the tree can be 
significant If many spectral classes and features arc needed to characterize 
the scene of Interest. Although the decision tree can become very compli- 
cated and awkward to use, this classifier is particularly well suited for 
use with mul titemporal or multitype data sets. 

Parallelling the complexity of Implementation as an Important variable 
In selecting a classification scheme Is the computational cost per 
classification. The computer time required per square kilometer for each 
segment and classifier Is shown In Table 5. In order of Increasing cost 
per square kilometer for classification, not Including cost for developing 
training statistics, were (1) MINIMUM DISTANCE (1.7 seconds), (2) ECHO 
(2.3 seconds), (3) LAYERED (2.3 seconds), (4) CLASSIFYPOINTS (3.8 seconds), 
and (5) CLASSIFY using ISOCLS statistics (11.3 seconds). 

IV. CONCLUSIONS 

The results of this study show little difference In the classification 
accuracies achieved by the five classification algorithms which were 
considered. However, the results for the CLASSIH' algorithm using two 
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Table 5. Computer CPU Time (seconds per square kiloiueter) Used by Each Classifier. 


u 

H 


H 

CO 

U 

H 







dlflerent training methods did show a difference, Indicating that the 
major variable affecting classification accuracy Is not the classifier, 
but the training method used la generating the class statistics to be 
used In the classification. The roost Important aspect of training Is 
that all cover types In the scene must be adequately represented by a 
sufficient number of samples In each spectral subclass. 

The ISOCLS training algorithm was a method designed for machine 
automation of a large portion of the training procedure. The statistical 
sampling method used for selection of training data Is theoretically sound, 
so it is possible that the lack of analyst refinement of the training 
statistics is seriously limiting the performance. Tlie clusters produced 
by this method are of mixed cover types which may adversely affect performance. 

Additional variables of interest in the study were complexity of 
use of the classifier and CPU cost per classification. Among the 
classifiers yielding similar classification accuracies, MINIMUM DISTANCE 
was the easiest for the analyst to use ..nd costs the least per classification. 

In conclusion, the classification performance of the five classification 
algorithms was found to be very similar when the same training method was 
utilized. The results suggest that development ol representative training 
statistics Is relatively more important for obtaining accurate classifi- 
cations chan selection cf the classification algorithm. 
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